Hey Michael Stapelberg, thanks for
creating i3 and making me believe again that
people have a clue and things can be done just right (specially when it comes
to something easily done crappy like window managers). Thanks for just getting
it all right, and not only from the software point of view, but also for the
community, the style around it, etc. Kudos.
Thanks to Axel Beckert (abe@), 12 people interested in Debian met last Tuesday
in Z rich and celebrated the start of our monthly Debian meetup.
New faces are always very welcome. If you live in Z rich, or if you re
visiting, please feel free to attend our meetup no registration necessary.
See the
initial announcement, and subscribe to community@lists.debian.ch for
updates.
Thanks to everyone for the nice evening, see you next time!
git repack -a -d will lead to a single packfile that optimally
contains HEAD, not caring about the history. Also, he showed me how to use
git cat-file. We did a small-scale experiment and the results were
promising. I told him I ll do the experiment of just committing the entire
unpacked source mirror to git and promised to follow up with the results, so
here goes:
stapelberg@couper ~/unpacked master $ time cat <<EOT git cat-file --batch >/dev/null :linux_3.2.32-1/sound/pci/ice1712/aureon.c :linux_3.2.32-1/sound/soc/codecs/sgtl5000.c :linux_3.2.32-1/sound/pci/hda/patch_conexant.c EOT cat <<<'' 0,00s user 0,00s system 62% cpu 0,006 total git cat-file --batch > /dev/null 4,38s user 1,08s system 99% cpu 5,477 total stapelberg@couper ~/unpacked master $ time cat linux_3.2.32-1/sound/pci/ice1712/aureon.c linux_3.2.32-1/sound/soc/codecs/sgtl5000.c linux_3.2.32-1/sound/pci/hda/patch_conexant.c >/dev/null cat linux_3.2.32-1/sound/pci/ice1712/aureon.c > /dev/null 0,00s user 0,00s system 69% cpu 0,006 total stapelberg@couper ~/unpacked master $ time cat <<EOT git cat-file --batch >/dev/null :i3-wm_4.6-1/libi3/font.c EOT cat <<<':i3-wm_4.6-1/libi3/font.c' 0,00s user 0,00s system 51% cpu 0,008 total git cat-file --batch > /dev/null 4,30s user 1,18s system 99% cpu 5,533 total stapelberg@couper ~/unpacked master $ time cat i3-wm_4.6-1/libi3/font.c >/dev/null cat i3-wm_4.6-1/libi3/font.c > /dev/null 0,00s user 0,00s system 0% cpu 0,004 totalEven after repeating those tests a couple of times to get a warm page cache, the result stays the same: git takes about 5 seconds to resolve the deltas in a large repository. Even if this was 5 seconds startup time and a very small amount of additional time per file, it would not be acceptable for our use case. The conclusion is that git is clearly not suitable for this kind of usage, which is not surprising after having heard a couple of times that git does not scale :-). For the curious, the
.git/objects directory is 29 GiB
for roughly 140 GiB of source code, so the delta encoding is quite impressive
in terms of saving space. However, keep in mind that the compressed (!) Debian
source archive is about 35 GiB, so the savings are not that huge.
I will arrive at DebConf 2013 on Sunday afternoon. In case you are interested
in Go (the programming language), systemd, i3 or getting your package reviewed,
please talk to me! :-)
Looking forward to meeting many of you in real life.
dh $@ --buildsystem=golang --with=golang. That s it!
Now, given that this package is very new and only two packages use it so far
(both of which are not yet in Debian, but real soon ), there are most likely
bugs and missing features. This is where you come in: when packaging software
written in Go, please contact me and tell me whether dh-golang worked for you
and what could be improved. In case something does not work, definitely report
it. Even if everything worked, I d be happy to have a look at your packaging
before you upload.
Also see https://wiki.debian.org/MichaelStapelberg/GoPackaging.
debdiff --controlfiles=ALL OLD_amd64.changes NEW_amd64.changesThen file a bug report which uses our usertag and CCs pkg-systemd-maintainers. See http://bugs.debian.org/713853 for an example. Thanks! Thank you for helping us and Debian in general!
Debsources integriert auch eine Code-Suche, allerdings werden lediglich die Quellen des Unstable-Zweigs durchsucht, der zirka ein Drittel des Quellcodes von Debsources ausmacht.This loosely translates to:
Debsources also integrates a code search engine, but it only searches the sources of the unstable tree, which makes up for roughly one third of the sources that Debsources covers.I suspect the author of the article arrived at this conclusion because Debsources talks about 400 GiB of source code, whereas Debian Code Search talks about 130 GiB of source code. However, it still struck me as odd and giving the wrong impression. I thought that packages don t differ that much between stable, testing and unstable. So I fired up
psql and pounded UDD
until it revealed how many packages are different between stable, testing and
unstable:
SELECT
COUNT(*)
FROM
(SELECT
migrations.source,
sources.version AS stable_version,
testing_version,
unstable_version
FROM sources
LEFT JOIN migrations
USING (source)
WHERE sources.release = 'wheezy') AS x
WHERE
regexp_replace(stable_version::text, E'-([0-9.]+)$', '') = regexp_replace(testing_version::text, E'-([0-9.]+)$', '');
procfs is not enough, /proc/$PID/exe needs to be
supported), which are currently only available on Linux. Point 15 of Lennart s
blog post 0pointer.de/blog/projects/the-biggest-myths.html
contains an incomplete list of these features.
Maintaining portable code increases complexity
Since systemd is written in C, the canonical way to write portable code is by
using conditional compilation, for example with ifdef
statements. That makes the code harder to understand and reason about, but
more importantly it blows up the test matrix. It
also requires any new change to be tested on all supported systems, which is
not feasible for most contributors. I think everybody agrees keeping complexity
low is a good thing.
What are the implications for our non-Linux ports?
We, the Debian project, have two realistic options in my point of view:
sources.d.n for short) that you will
probably find interesting. sources.d.n follows closely the Debian archive in
two ways:
*-updates (ex volatile),
*-proposed-updates, and *-backports (from Wheezy on)sources.d.n you can therefore browse the content of Debian source
packages with usual code viewing features like syntax highlighting. More
interestingly, you can search through the source code (of unstable only,
though) via integration with http://codesearch.debian.net. You can also use
sources.d.n programmatically to
query available versions or
link to specific lines, with the
possibility of adding contextual pop-up messages
(example).
In fact, you might have stumbled upon sources.d.n already in the past few
days, via other popular Debian services where it has already been integrated.
In particular: codesearch.d.n now defaults to show results via sources.d.n,
and the PTS has grown new "browse source
code" hyperlinks that point to it. If you've ideas of other Debian services
where sources.d.n should be integrated, please let me know.
I find Debsources and sources.d.n already quite useful but, as it often
happens, there is still a lot
TODO.
Obviously, it is all Free Software (released under GNU AGPLv3). Do not hesitate
to report new bugs and, better, to submit patches for the outstanding ones.
Acknowledgements
sources.d.n wouldn't exist without him.sources.d.n with other
services, in particular:codesearch.d.n integration), andsources.d.nsources.d.n requires ~381
GB of disk space to hold all uncompressed source packages, plus ~83 GB for
the local (compressed) source mirror
sources.d.n for short) that you will probably find
interesting. sources.d.n follows closely the Debian
archive in two ways:
*-updates
(ex volatile), *-proposed-updates, and
*-backports (from Wheezy on)sources.d.n you can therefore browse the
content of Debian source packages with usual code viewing features
like syntax highlighting. More interestingly, you
can search through the source code (of unstable
only, though) via integration with http://codesearch.debian.net.
You can also use sources.d.n programmatically to
query available
versions or link to specific
lines, with the possibility of adding contextual
pop-up messages (example).
In fact, you might have stumbled upon sources.d.n
already in the past few days, via other popular Debian services
where it has already been integrated. In particular:
codesearch.d.n now defaults to show results via
sources.d.n, and the PTS has grown new "browse
source code" hyperlinks that point to it. If you've ideas of other
Debian services where sources.d.n should be
integrated, please let me know.
I find Debsources and sources.d.n already quite
useful but, as it often happens, there is still a lot
TODO. Obviously, it is all Free Software
(released under GNU AGPLv3). Do not hesitate to report new bugs
and, better, to submit patches for the outstanding ones.
Acknowledgements
sources.d.n wouldn't exist without him.sources.d.n with other services, in particular:
codesearch.d.n integration), andsources.d.nsources.d.n requires ~381 GB of disk
space to hold all uncompressed source packages, plus ~83 GB for the
local (compressed) source mirror
/sbin/init, we can make it the default init system. If and when
that happens is a different matter and it s not necessary for all packages to
have systemd support.
sysvinit compatibility
systemd natively supports sysvinit scripts, meaning your existing package will work as-is but you cannot utilize all the features that systemd provides. The sysvinit support works very well, as you can try in a fresh Debian wheezy VM. In the output of systemctl list-units , every entry which has an LSB: prefix is actually a sysvinit script.
The mechanism with which systemd decides whether to use an init script or a
service file is by looking whether a service file with a corresponding name
exists. That is, if e.g. apache2.service exists, systemd will
prefer it over /etc/init.d/apache2.
To make this crystal clear: it is not necessary to ship service files for all
services in some kind of flag day.
systemd supports a mixed installation where some services use init scripts
and some services use service files.
Adding systemd support to your package
In a nutshell, it usually works like this:
/lib/systemd/system/foo.service..service file.debian/package.serviceapache2.service for /etc/init.d/apache2)
--with=systemd in debian/rules and Build-Dep on dh-systemd
/sbin/init and systemd will not be enabled right away. It is only
after you specify the kernel parameter init=/bin/systemd in
/etc/default/grub that you switch to systemd. In case you want to
go back, simply boot without this kernel parameter.
Conclusion
In conclusion, the transition is straight-forward and the necessary
infrastructure is in place. systemd is available in Debian and can be used
today. Packages can add systemd support whenever their maintainer(s) feel like
it. There is no need for a flag day. We
can switch the default whenever we think we are ready.
mplayer
gpn13-debian-code-search.mp4 and press v to enable subtitles.
I intend to eventually put the (subtitled) video on YouTube and refer to it
from codesearch.debian.net, but I
wanted to post the video in its current form already.
The presentation itself explains the motivation behind Debian Code Search and
how it works. You don t need any knowledge of the system in order to understand
the talk. Enjoy!
systemd has too many dependencies, or it is bloated, or it does too many things, or is too complexNow this concern actually has a lot of different facets, and I am trying to share my opinion on each of them. systemd has too many dependencies First, let s start with too many dependencies , because that is easy to check and reason about. I have created a document which lists all dependencies of the systemd binary itself (pid 1) and all the binaries which are currently shipped by the systemd Debian package. If you don t want to take my word for granted, please read that document. Have you read the document? Very nice! As you can see, the systemd binary itself has 10 dependencies (excluding libc). Now, the question is, what is bad about dependencies? Why do people list dependencies as a top concern?
enforce_user
which uses libcap. You ll notice that the code is not complex and usage of
the libraries is clear.
Software bloat is a process whereby successive versions of a computer program become perceptibly slower, use more memory or processing power, or have higher hardware requirements than the previous version whilst making only dubious user-perceptible improvements.The first part of the definition certainly does not match systemd it is measurably faster than sysvinit. As for memory usage: systemd s RSS is 1.8 MiB, whereas sysvinit uses 0.8 MiB. As I argued on the More memory use point in the dependencies section, I think the additional resource cost is well worth the benefits. Also note that systemd s features are NOT all implemented in the binary which is PID 1. As explained in the dependency list, systemd consists of many cleanly separated binaries. So if a new version of systemd gathers an additional feature, this does not mean that your PID 1 will be bigger. While systemd runs on any hardware, it has an indirect hardware requirement: it requires some Linux kernel features (which are all enabled in Debian kernels). That might rule out usage of systemd on really old embedded hardware where you don t have a chance to update the kernel. While it is sad that those machines cannot profit from systemd, switching to systemd as a default has no downside either: Debian continues to support sysvinit for quite some time, so these machines will continue to work even with upcoming Debian versions. systemd does too many things The Wikipedia definition continues:
[ ] perceived bloat can occur from the software servicing a large, diverse marketplace with many differing requirements. Most end users will feel they only need some limited subset of the available functions and will regard the others as unnecessary bloat, even if people with different requirements do use them.I think the last part of the Wikipedia definition applies to systemd: it does service a large and diverse marketplace . That marketplace is the entirety of existing software which is started by an init system. Also, systemd can be used on a wide range of hardware (embedded devices, tablets, phones, notebooks, desktops, servers) which requires different features. As an example: on a desktop system you typically don t care strongly about a watchdog feature, but on embedded or servers that feature is very handy. Similarly, on a tablet, forward secure sealing of logfiles is not as important as on a server. Therefore, I can understand if you feel that you don t need many of the features systemd provides. But please think of other users and maintainers who are very happy with systemd s benefits. Also note that while systemd supports many things (in separate binaries!), you don t have to use them all. It still makes sense to ship them all in the same package. Take coreutils as another example in that area. The binaries belong together, even though you most likely haven t used all of them (e.g. od, pr, ptx, :-)). systemd is too complex The remaining concern is that systemd is too complex. In my experience, complexity is often inherent to a specific area and one cannot simply make it go away. Instead, there are different models of how that complexity is represented. Think of the monolithic Linux kernel versus the MINIX microkernel. The latter has a very small amount of lines of source in the kernel, but puts the complexity into userspace. The former uses a different approach with more source in the kernel. The arguments between both camps show that neither is clearly right or clearly wrong. In a way, sysvinit represents the MINIX model: it has a small core (the init binary itself), but a lot of complexity in shell scripts and external programs. The fact that solutions are copied from one init script to another leads to lots of subtle errors and makes code reuse really hard. systemd however has more source code in the binaries, but requires only very simple, descriptive, textual configuration instead of complex init scripts. To me, it seems preferable to have the complexity in a single place instead of distributed across lots of people and projects. Conclusion In a way, you are right. systemd centralizes complexity from tons of init scripts into a single place. However, it therefore makes it very easy for maintainers to write service files (equivalent of an init script) and provides a consistent and reliable interface for service management. Furthermore, it is different than sysvinit, and different solutions often seem complex at first. While systemd consumes more resources than sysvinit, it uses them to make more information available about services; its finer-grained service management requires more state-keeping, but in turn offers you more control over your services.
Next.